View Dependent Layered Projective Texture Maps

International audienceCapturing and rendering of real scenes in an immersive virtual environment is still a challenging task. We present a novel workflow, using high-quality, view-dependent projective texturing at low texture memory cost, while reinforcing artist's control over image quality. Photographs of a real scene are first used to create a 3D model with standard tools. Our method automatically orders geometry into optimized visibility layers for each photograph. These layers are subsequently used to create standard 2D image-editing layers, enabling artists to fill in missing texture using standard techniques such as clone brushing. The result of this preprocess is used by our novel layered projective texture rendering algorithm, which has low texture memory consumption, high interactive image quality and avoids the need for subdividing geometry for visibility. We show results of our implementation on a real-world project

Textures projectives à calques dépendantes du point de vue

International audienceLa capture et le rendu des scènes réelles dans le contexte des environnements virtuels immersifs sont encore des tâches très difficiles. Nous présentons une nouvelle approche, en utilisant des textures projectives dépendantes du point de vue, de haute qualité et avec un faible coût en mémoire texture. Dans un premier temps, des photographies de la scène réelle sont utilisées pour créer un modèle 3D avec des outils standards. Notre méthode ordonne automatiquement la géométrie en groupes à niveaux de visibilité différents pour chaque point de vue. Ces groupes sont ensuite utilisés pour découper les images en calques, permettant aux artistes le remplissage des textures manquantes, dues à leur non-visibilité, en utilisant des outils comme le tampon de duplication (clone brush). Le résultat de ce processus est utilisé par notre nouvel algorithme de rendu en utilisant des textures projectives à calques, ayant un faible coût d'utilisation de mémoire texture, une haute qualité de rendu et dans le cas de textures projectives dépendantes du point de vue, évite la subdivision de la géométrie engendrée par la visibilité en utilisant des méthodes antérieures

Design and Evaluation of a Real-World Virtual Environment for Architecture and Urban Planning

International audienceIn this paper we present a user-centered design approach to the development of a Virtual Environment (VE), by utilizing an iterative, user-informed process throughout the entire design and development cycle. A preliminary survey was first undertaken with end users, that is, architects, chief engineers, and decision makers of a real-world architectural and urban planning project, followed by a study of the traditional workflow employed. We then determined the elements required to make the VE useful in the real-world setting, choosing appropriate graphical and auditory techniques to develop audiovisual VEs with a high level of realism. Our user-centered design approach guided the development of an appropriate interface and an evaluation methodology to test the overall usability of the system. The VE was evaluated both in the laboratory and, most importantly, in the users' natural work environments. In this study we present the choices we made as part of the design and evaluation methodologies employed, which successfully combined research goals with those of a real-world project. Among other results, this evaluation suggests that involving users and designers from the beginning improves the effectiveness of the VE in the context of the real world urban planning project. Furthermore, it demonstrates that appropriate levels of realism, in particular spatialized 3D sound, high-detail vegetation, and shadows, as well as the presence of rendered crowds, are significant for the design process and for communicating about designs; they enable better appreciation of overall ambience of the VE, perception of space and physical objects, as well as the sense of scale. We believe this study is of interest to VE researchers, designers, and practitioners, as well as professionals interested in using VR in their workplace

A User-Centered Approach on Combining Realism and Interactivity in Virtual Environments

International audienceIn this paper we describe a project that adopts a usercentered approach in the design of virtual environments (VEs) with enhanced realism and interactivity, guided by real-world applications in the areas of urban planning/ architecture and cultural heritage education. In what concerns realism, we introduce an image-based 3D capture process, where realistic models are created from photographs and subsequently displayed in a VR system using a high-quality, view-dependent algorithm. The VE is further enhanced using advanced vegetation and shadow display algorithms as well as 3D sound. A high degree of interactivity is added, allowing users to build and manipulate elements of the VEs according to their needs, as specified through a user task analysis and scenario-based approach which is currently being evaluated. This work is developed as part of the EU-funded research project CREATE

Image-based Techniques for the Creation and Display of Photorealistic Interactive Virtual Environments

International audienceIn this paper we introduce an image-based 3D capture process for the creation and display of photorealistic virtual environments (VEs). The resulting VEs aim to realistically recreate existing real-world scenes that can be displayed in a range of immersive VR systems using a high-quality, view-dependent algorithm and further enhanced using advanced vegetation, shadow display algorithms and 3D sound. The scenes, an archaeological site and an urban environment, were chosen according to real-world applications in the areas of urban planning/architecture and cultural heritage education. The users in each case are able to reconstruct or manipulate elements of the VEs according to their needs, as these have been specified through a detailed user requirements survey. Furthermore, a user task analysis and scenario-based approach has been adopted for the design of the virtual prototypes and the evaluation, which is currently underway. This work is being developed in the context of the EU-funded research project CREATE and the first examples of the prototype system in use are described and demonstrated in this paper

Automatic generation of consistent shadows for Augmented Reality

Sponsor : CHCCS The Canadian Human-Computer Communications SocietyInternational audienceIn the context of mixed reality, it is difficult to simulate shadow interaction between real and virtual objects when only an approximate geometry of the real scene and the light source is known. In this paper, we present a realtime rendering solution to simulate colour-consistent virtual shadows in a real scene. The rendering consists of a three-step mechanism: shadow detection, shadow protection and shadow generation. In the shadow detection step, the shadows due to real objects are automatically identified using the texture information and an initial estimate of the shadow region. In the next step, a protection mask is created to prevent further rendering in those shadow regions. Finally, the virtual shadows are generated using shadow volumes and a pre-defined scaling factor that adapts the intensity of the virtual shadows to the real shadow. The procedure detects and generates shadows in real time, consistent with those already present in the scene and offers an automatic and real-time solution for common illumination, suitable for augmented reality

Participatory Design and Evaluation of a Real-World Virtual Environment for Architecture and Urban Planning

In this report we present a participatory design approach to the development of a VE, with an iterative, user-informed process throughout the entire design and development cycle. A preliminary survey was first undertaken with end-users, i.e., architects, chief engineers and decision makers of a real-world urban planning project, followed by a study of the traditional workflow employed. We then determined the elements required to make the VE useful in the real-world setting, choosing appropriate elements (image-based textures for facades, realistic vegetation, shadows, 3D sound and crowds) to develop a rich and audiovisually realistic VE. Our participatory design approach guided the development of an appropriate interface and an evaluation methodology to test the overall usability of the system. The VE was evaluated both in the laboratory and in the users' natural work environments. The results suggest that participatory design can help ensure that the VE will meet the practical needs of its end-users; that appropriately chosen multiple views of the scene are useful in urban planning, and that the realism enhancements introduced increase the effectiveness of VEs for the chosen context

Volumetric Reconstruction and Interactive Rendering of Trees from Photographs

International audienceReconstructing and rendering trees is a challenging problem due to the geometric complexity involved, and the inherent difficulties of capture. In this paper we propose a volumetric approach to capture and render trees with relatively sparse foliage. Photographs of such trees typically have single pixels containing the blended projection of numerous leaves/branches and background. We show how we estimate opacity values on a recursive grid, based on alphamattes extracted from a small number of calibrated photographs of a tree. This data structure is then used to render billboards attached to the centers of the grid cells. Each billboard is assigned a set of view-dependent textures corresponding to each input view. These textures are generated by approximating coverage masks based on opacity and depth from the camera. Rendering is performed using a view-dependent texturing algorithm. The resulting volumetric tree structure has low polygon count, permitting interactive rendering of realistic 3D trees. We illustrate the implementation of our system on several different real trees, and show that we can insert the resulting model in virtual scenes

Capture et Rendu à Base d'Images avec Application aux Projets Urbains

The first author is funded by a CIFRE doctoral fellowship, in partnership with the Centre Scientifiqueet Technique du Batiment (http://www.cstb.fr). This research presented in this theses werepartially funded by the EU IST project CREATE, IST-2001-34231, http://www.cs.ucl.ac.uk/create.ImageModelerTM was gracefully provided by REALVIZ in the context of the CREATE project.This thesis addresses the capture real objects to be integrated in a virtual environment and their efficient display. The thesis has two parts; in the first part, we present two new methods for the capture and rendering real objects using image-based techniques, and in the second part, we present an complete virtual reality system that allows the use of different realistic techniques for the design of urban projects, as well as a first evaluation of this system. The first method proposed allows the capture of a real scene with relatively simple geometry, for example building façades. The capture is done using an small number of photographs. With this method the captured objects have a compact representation; the work of the computer graphics artist is simplified to allow higher quality render and the rendering method does not need geometric subdivision as in previous methods. The second method proposed is for the capture of complex objects, in our case trees, using as input a series of photographs taken around the object. A first estimation of opacities is done with a method inspired by tomography, that give us a volumetric representation of the tree. The high frequency detail of the leaves is added by an algorithm of texture generation using the original images. The result is a very realistic rendering of real captured trees, that can be added in a virtual environment. The virtual reality system presented in the second part of the thesis integrates image-based rendering methods, and other algorithms for the realistic rendering (shadows, 3D sound, etc.). The multi platform system works on systems such as the Workbench of the INRIA Sophia-Antipolis or the RealityCenter of the CSTB. Our work on this real world project, the construction of the Tramway of Nice (France), allowed us to evaluate the choices on realism and the interface of the system. The architects in charge of the Tramway project on the Garibaldi square in Nice performed a set of test, and we present here the first conclusions of our observation.Cette thèse traite le problème de la capture des objets réels dans le but de les intégrer dans un environnement de réalité virtuelle et de les afficher (ou les rendre) d'une manière efficace. La thèse a deux parties : dans la première partie, nous présentons deux nouvelles méthodes pour la capture et le rendu des objets réels avec des techniques à base d'images, et dans la deuxième, nous présentons un système intégré de réalité virtuelle qui permet l'utilisation de plusieurs techniques réalistes pour la conception et l'évaluation de projets d'urbanisme. La première méthode proposée permet la capture d'objets réels avec une géométrie relativement simple, comme les façades de bâtiments par exemple. La capture se fait à partir d'un faible nombre de photographies. Avec cette méthode les objets capturés ont une représentation compacte ; le travail des infographistes est facilité pour permettre un affichage de meilleure qualité et enfin la méthode de rendu ne nécessite pas un découpage géométrique des objets, comme c'était le cas pour les méthodes précédentes. La deuxième méthode proposée traite le cas des objets complexes comme les arbres, à partir d'un ensemble de photographies prises autour de l'objet. Une première estimation de l'opacité est effectuée avec une méthode inspirée de la tomographie, ce qui donne une représentation volumique de l'arbre. Ensuite le détail de haute fréquence du feuillage est ajouté par un algorithme de génération de textures à partir des images d'origine. Le résultat est l'affichage très réaliste des arbres réels capturés, qui peuvent être insérés dans des environnements virtuels. Le système de réalité virtuelle présenté dans la deuxième partie de la thèse intègre des méthodes d'affichage à base d'images, ainsi que d'autres algorithmes d'affichage réaliste (ombres, son 3D, etc.). Le système multi plateforme est opérationnel sur des systèmes de réalité virtuelle allant du plan de travail virtuel à l'Inria Sophia-Antipolis jusqu'à la salle RealityCenter du CSTB. Notre travail dans le cadre d'un projet réel, celui du Tramway de Nice, nous a permis d'évaluer nos choix sur le réalisme et l'interface du système. Plusieurs essais ont été effectués avec les architectes responsables du réaménagement de la Place Garibaldi à Nice, et nous présentons ici les premières conclusions de nos observations